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@ A video processing system for movement simulation. 

@ A video processing system for use in creating three 
dimensional movement of a video Image on a display screen 20. 
At least one set of axes is displayed on the display screen and a 
computer 24, in response to signals from a user operated stylus 
21 and touch tablet 22, controls the manipulation of image data 
in an Image store 26 to control rotational or translattonal 
movement of the Image, with respect to an axis selected by the 
user. In the described embodiment several sets of linked axes 
may be defined. Rotational and translatlonal movement with 
respect to an axis is controlled respectively by selecting a 
marker identifying the axis or by selecting a line representing 
the axis. Selection is made by way of a cursor under the control 
of the stylus and touch tablet The system can also display a 
locus representing the user defined movement of e.g. the 
centre of the Image. 
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Description 

A VIDEO PROCESSING SYSTEM FOR MOVEMENT SIMULATION 



FIELD OF THE INVENTION 

The present Invention relates to a video process- 5 
ing system having a facility for simulating three 
dimensional movement of a live video picture. 

BACKGROUND OF THE INVENTION 

Video processing machines for simulating three 10 
dimensional movement of a live video picture are 
known. Hitherto, the controls for such machines 
have included switching devices for selecting oper- 
ations and movement detectors, such as levers and 
tracker balls, for defining the temporal nature of the 15 
selected effect. These controls provide a means by 
which a user of the machine can among other things 
achieve the effect of apparent three dimensional 
movement of for example Qve video pictures on a 
display screen. However, such controls are limited In 20 
that multiple actuations of different levers and 
switches together with movement of the tracker bail 
are required In order to achieve the desired effect. 
Thus, efficient use of the machine requires a high 
level of experience and dexterity on the part of the 25 
user and, whilst highly satisfactory effects are 
achievable using these controls, an undue amount of 
time can be spent by the user in achieving desired 
results. 

30 

OBJECTS AND SUMMARY OF THE INVENTION 

The principal object of the present invention is to 
provide a solution to the above and associated 
limitations and to provide an improved video pro- 
cessing system for image manipulation to simulate 35 
movement. 

The invention generally resides in the concept of 
providing an improved video processing system in 
which a set of axes can be displayed over an image 
and the image can be manipulated by reference to 40 
said axes by a user by way of input means to realise 
the desired simulated movement effect. 

According to the invention in one of its aspects 
therefore there Is provided a video processing 
system for use in creating apparent three dimen- 45 
sionai movement of a video image on a display 
screen, In which system at least one set of axes is 
displayed and processing means responsive to user 
operable input means are provided for processing 
image data to control apparent rotational movement 50 
of the image about, and/or apparent translational 
movement In the direction of, a user selected axis of 
said set. 

In an embodiment of the invention, to be de- 
scribed In greater detail hereinafter, a plurality of 55 
sets of axes are available to be selectively displayed 
and individual axes within selected sets can be 
selected by the user by way of a cursor displayed on 
the display screen, the position of the cursor being 
controlled by use of the user operable Input means, 60 
namely in the embodiment in question a pressure 
sensitive stylus and an associated touch tablet. The 
spatial relationship between the various sets of axes 



can be defined by the user by way of the cursor 
displayed on the display screen and manipulated by 
the user with his stylus and touch tablet By 
providing a plurality of sets of axes and providing for 
selective movement in each axes set, the system 
provides a means by which different levels of 
movement of an Image may be achieved as will 
become apparent from the following. 

A first level set of axes will normally be provided at 
the centre of a picture, that is to say at the centre of 
a framed Image, and the system arrangement 
enables the user to rotate the picture with any axis of 
such first set of axes or to move the picture with the 
origin of the axes set along the One of any axis as 
required. The first set of axes Is normally set such 
that two of the axes are arranged to He parallel to the 
x and y directions of the picture (and of the display 
screen, assuming the picture edges are parallel to 
the edges of the screen which will normally be the 
case) and a third axis is arranged to be perpendicu- 
lar to the picture (and to the screen). Translation of 
the picture along the x and y axes, when the x and y 
axes of the picture are parallel to the x and y axes of 
the display screen, will simply cause the picture to 
shift left or right and up or down on the display 
screen, while translation along the z-axis will cause 
the size of the picture to change as It moves in 3D 
space away from or towards the display screen. The 
effect of rotation of the picture about an axis needs 
no explanation. 

Higher level sets of axes are also made available 
to be utilised successively, and can be located by 
the user at any desired position in three dimensional 
space remote from the first axis location. The origin 
of the first set of axes, also called the picture level 
axes, (i.e. the centre of the Image) is maintained at a 
fixed distance in three dimensional space from the 
origin of the second level axes and In this way 
rotation and translation of the picture in relation to a 
second level axis will cause movement of the picture 
In three dimensional space as though the centre of 
the picture were fixedly connected to one end of a 
rigid arm having Its other end located at the origin of 
the second level axes and movable therewith in 
rotation and translation. Thus, translation and rota- 
tion of the second level axes may be effected to 
change the shape and size of the picture to 
accommodate changes in perspective and viewing 
position as the image is moved. Further higher levels 
of axes may be defined at other points in three 
dimensional space as required. Each level of axes 
will be maintained at a fixed distance from the next 
higher level axes and in this way complex move- 
ments of the picture through three dimensional 
space may be defined by the user. 

As will be explained in the following, in use of the 
system of the Invention to simulate three dimen- 
sional movement of a video picture, the first or 
picture level set of axes is automatically displayed at 
the centre of the picture when the system Is 
activated. The user can then translate and rotate the 
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picture level set of axes as required by use of his 
stylus and touch tablet When a second Irvel set of 
axes is required this Is derived from the picture level 
set of axes which is, in effect, dragged out to the 
required origin position of the second set of axes, 
again by use of the stylus and touch tablet. The 
second set of axes as thus established are thus 
oriented identically to the orientation of the picture 
level set of axes at the time when the second set of 
axes are set up. Once the second set of axes has 
been set up, the user has the facility to rotate and 
translate In the second axes with corresponding 
movement of the picture. Subsequent axes sets are 
similarly established, each subsequent axes set 
being dragged out from the preceding set as above 
described in the case of the second and picture level 
set of axes. Ail axes sets that are set up by a user 
remain displayed and the user has the option to work 
In any selected one of a series of sets of axes thus 
established. 

In order to facilitate use of the system each set of 
axes can be displayed as a set of lines representing 
the orientation of each axis in the set and accompa- 
nying labels which identity each axis, the system 
being arranged such that selection of a label by 
cursor control enables rotation about the associated 
axis to be effected and selection of a line enables 
translational movement along the associated axis to 
be effected. 

Furthermore, the system may be configured so as 
to enable the display of a locus representing the 
movement of the picture in three dimensional space 
and/or representing the movement of the origin of a 
user selected set of axes in three dimensional 
space. The video picture may comprise a sequence 
of video frames and the system can be arranged 
such that at least some of the frames can be defined 
by the user as key frames which are displayed at 
user defined position within the three dimensional 
movement. The locations of other frames in the 
sequence not having a user defined location can be 
calculated automatically by extrapolation between 
user defined keyframes. In this case the locations of 
keyframes in the movement are represented by first 
markers In the locus displayed on the display screen, 
and the locations of other frames are represented by 
second markers In the locus. 

Thus according to another aspect of the invention 
there is provided a video processing system com- 
prising frame storage means, data processing 
means, a viewing screen and user operable means 
arranged for co-operation with said processing 
means to effect manipulation of frame store data to 
simulate movement of a video Image in three 
dimensional space, and wherein the system further 
comprises means for storing data defining the 
nature of said manipulations, means for determining 
address data representative of the positions of a 
predetermined element of a manipulated video 
image, and means for displaying connections bet- 
ween said positions to show the movements of said 
element. 

Further features of the Invention are as set forth in 
the appended claims and, together with those 
features above mentioned, will become clear from 
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consideration of the detailed description of an 
exemplary embodiment of the invention which will be 
given in the following with reference to the accompa- 
nying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a video processing system 
embodying the present Invention; 
w Figure 2 shows an example of a typical 

display generated while using the system of 
Figure 1 ; and 

Figure 3 shows a locus representing move- 
ment of a point In the picture or the origin of a 
15 set of axes. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
A video processing system embodying the pres- 
ent invention is shown in Figure 1 as comprising a 

20 television type monitor 20, a pressure sensitive 
stylus 21 and a touch tablet 22 each connected to a 
processing system 23. The processing system 23 
Includes a control computer 24, an address transfor- 
ming circuit 25, an image framestore 26, a graphics 

25 framestore 27 and a video combiner 28. In operation 
an input signal is supplied by an Input device 29 
which for live video from a camera or an output from 
an analog video tape recorder will include an analog 
to digital converter. Alternatively the input may be in, 

30 say, D1 digital format, an approved digital standard 
for making digital video tape recordings from tape, 
disc or other digital processing system. 

The processing system is arranged under soft- 
ware control 23 to enable video signals supplied 

35 thereto via the input device 29 to be manipulated 
frame by frame in real time to produce special 
rotational and translational effects by writing pixels 
to the image framestore 26 in a first order and 
reading them from the framestore 26 in a different 

40 order. Image manipulation systems which operate in 
this way are known and generally work as either read 
side systems. In which Image data is written Into a 
store In normal raster order and then read from the 
store for display in a different order, or write side 

45 systems, in which the order of the data is changed as 
it is written to the store and then Is read convention- 
airy. One example of a machine which writes to the 
store In normal raster order and then reads in a 
different order (read side) is manufactured by the 

50 present Assignee and sold under the trade mark 
"ENCORE" and is described in European Patent 
Publication No. EP-A-0283159 and In corresponding 
US Patent Application No. 159780 and Japanese 
Patent Application No. Sho 63- 62099 the disclo- 

55 sures whereof are incorporated herein by way of 
reference. Another machine which performs its 
manipulations while writing to a framestore and then 
reads In a conventional way (write side) is manufac- 
tured by the present Assignee and sold under the 

60 trade mark "MIRAGE" and is described in UK Patent 
Applications Nos. 2119594 and 2158671 the disclo- 
sures whereof are incorporated herein by way of 
reference. Write side processing is generally the 
more difficult, and the described embodiment of the 

65 present invention as represented in Figure 1 is 
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configured as a read side system though the 
invention could also be embodied in a write side 
system. 

The system according to the present invention 
does not require the control panel having switch 
buttons and a tracker ball that are provided In our 
earlier ENCORE machine but instead Is arranged 
under software control so that all operations can be 
effected by use of the pressure sensitive stylus 21 
upon the touch tablet 22. More particularly, in the 
system of the present Invention, a movement of the 
stylus 21 on the touch tablet causes the touch tablet 
to output address signals representing movements 
in x and y directions as shown by the axes 30 in 
Figure 1 . An additional parameter of the movement is 
defined by the pressure that is applied to the stylus 
as sensed by means of a pressure transducer 
incorporated info the stylus, for example by applica- 
tion of stylus pressure In an on/off fashion to define 
start and end points of an operation, e.g. the origin 
positions of successively established sets of axes. 
The stylus 21 and touch tablet 22 enable the user to 
control the movement of a cursor on the display 
screen and in this way a user is able to select an 
operation, effect it and store the result without 
looking away from the monitor 20. The vast number 
of picture movements achievable with a system of 
this type can be combined with previously known 
image processing features or options such as sizing, 
corner pinning etc selected from a menu displayed 
on the monitor. 

Movement of the stylus swiftly across the edge of 
the touch tablet (a process known as swiping) is 
arranged in per se known manner to call up a menu 
of user selectable options which can be displayed 
anywhere in the screen, for example In the lower, or 
upper, quarter of the screen. The menu may obscure 
the display area below it or alternatively it may be 
made transparent. As the stylus is placed onto the 
touch tablet a cursor is displayed on the monitor and 
is controlled (moved) by moving the stylus on the 
touch tablet. An operation is selected by the user by 
moving the stylus so that the cursor appears over 
the appropriate menu item and then pressing the 
stylus onto the touch tablet so as to cause the 
pressure transducer to output a signal to indicate to 
the system the selection of the corresponding menu 
option. The system according to the present 
invention is therefore arranged by its software to be 
controlled in a very similar way to the video graphics 
system manufactured by the present Assignee and 
sold under the trade mark "PAINTBOX". 

As can be seen from the exemplary showing of 
Figure 2 the image 33 read from store 26 has been 
reduced in size so that it occupies only part of the 
full screen of monitor 20. The system is arranged 
such that transtatlonal Image movements can be 
controlled by operation of the stylus on the touch 
tablet so as to position the cursor over an x, y or z 
letter associated with a particular set of axes. With 
the cursor so positioned, pressing the stylus on the 
touch tablet (and thereby activating the pressure 
transducer therein) causes the respective axis to be 
selected and thus allows movement in the selected 
axis direction to be effected by further movement of 



the stylus. Similarly identifying an axis by placing the 
cursor over the line representing it and pressing the 
stylus to Indicate selection allows rotation about that 
axis to be controlled by the user. Since the origin of 
5 the first set of axes 31 is always at the centre 32 of 
the picture, movement In relation to any point In 3D 
space other than in relation to the centre 32 of the 
picture requires the operator to ascend to a higher 
level, I.e. a different set of axes 34, 36, 37. 

10 The on screen menu (not shown) provides for the 
selection of higher level axes. On selecting the 
second level, say, a second level reference point 
(origin) 34, with its accompanying set of axes 
oriented identically to the picture level axes 31, may 

15 be positioned anywhere on the screen by use of the 
stylus. By further use of the stylus In a manner as 
above described for the picture level axes 31 
movements In this second set of axes can be 
effected. At this second level the user Instructions 

20 define movements of the origin of the picture level 
axes 32 about the origin of the second level axes 34. 
Similarly, at the third level movements of the origin of 
the second level axes 34 are defined with respect to 
the origin of the third level axes 36, and at the fourth 

25 and in this case highest level movement of the origin 
of the third level axes 36 is defined with respect to 
the origin of the fourth level axes 37. 

Figure 2 shows the way In which picture move- 
ments may be controlled at different hierarchical 

30 levels by the use of a series of linked axes. On the 
display screen 20 a first set of axes 31 referred to 
hereinafter also as picture level axes, are automati- 
cally set up at the centre 32 of a picture 33 In 
response to user initiation of the system to enable 

35 the user selectively to rotate the picture with, or 
translate the picture In the direction of, any of the 
axes 31 x, 31 y, 31 z, such rotation and/or translation 
being effected conventionally as hereinbefore de- 
scribed. Two of the axes 31x, 31y lie parallel to the x 

40 and y directions of the picture 33 and Initially (before 
any picture rotation) of the display screen 20, and 
the third axis 31z Is arranged to be perpendicular to 
the picture and thus initially (before any picture 
rotation) to the display screen 20. Each axis 31 x, 31y, 

45 31z is represented on the display screen by a line 
Indicating the orientation of the axis and a letter x, y 
or z identifying the axis, and image manipulation is 
effected by operation of the stylus/touch tablet to 
move a cursor on the display screen and by means 

50 of the cursor to select either rotation or translation 
relative to a selected axis. Higher levels of axes may 
also be utilised. A menu (not shown) that is provided 
on the display screen enables the user to select a 
higher level axes set, for example second level axes 

55 34, and once selected the axes 34 can be located by 
use of the stylus and its pressure sensitive facility on 
the touch tablet at any point in three dimensional 
space and thereafter rotation and/or translation in 
the second axes level can be effected. The distance 

60 In three dimensional space between the origin 32 of 
the picture level axes 31 and the origin of the second 
level axes 34 Is held fixed once it has been set by the 
user, and is represented by the broken line 35 in the 
drawing. Further higher levels of axes, for example 

65 third level axes 36 and fourth level axes 37, may also 
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be defined at other points in three dimensional 
space If so required, the origins of such exes being 
held respectively at user defined fixed distances, 
represented In Figure 2 by broken lines 38, 39, from 
the origins of the axes at the level below. Image 
manipulation by rotation and/or translation relative 
to any axis of the successive hierarchical levels of 
user defined axes Is effected exactly as above 
described for the picture level axes. 

The picture manipulation movements achievable 
by the present invention may be likened to the 
behaviour of a multiply articulated arm moving 
through three dimensional space. This notional 
articulated arm would have three links correspond- 
ing to the broken Ones 35, 38 and 39 connected by 
ball joints at the origins of the axes 34 and 36 and 
with the centre 32 of the picture rotatably connected 
to the link 35 by a ball joint at the origin of axes 31 
and the link 39 also connected to a ball Joint at the 
origin of the axes 37. Since the articulated arm Is 
merely notional, the links are such as to allow total 
movement of the picture 36 apparently in three 
dimensional space, including the situation of two 
links of the arm occupying the same space. The link 
system is set up by the system user who determines 
the initial positions of the bail joints 34, 36, 37 and the 
lengths of the links 35, 38, 39 and the user Is then 
free to define movements of the picture by rotations 
and translations within the link system. 

In order to facilitate operation of the system, each 
level of operation (i.e. each set of axes and 
associated graphics) advantageously has an associ- 
ated color relating to that level so that movements in 
any given level are represented in Its particular color. 
Thus the axes 31 for movement at the picture level 
are displayed In a picture level color, and the higher 
level axes 34, 36, 37 are displayed in different colors. 

During the creation of a move, l.e. 3D movement of 
the picture, only a single picture frame 33 Is read 
from the image store 26 and displayed on the 
monitor 20 and this picture frame 33 remains on the 
monitor's display whilst a move is being created. 
Video data for the or each set of axes 31 , 34, 36, 37, 
and any other Information to be displayed over the 
picture, for example the cursor, Is stored In the 
graphics framestore 27 which Is addressed In 
synchronism with the image store 26. However once 
the move has been defined, sequential frames of live 
video etc may be written to the store 26 via input 
device 29. The computer 24 includes a memory 
device for storing a sequence of manipulated read 
address signals representing the move. As the move 
progresses a narrow band of colour can usefully be 
arranged to move across the top of the screen to 
inform the operator how much memory space is 
avaitable, and in this way all the information required 
by the operator is displayed on the monitor 20. The 
apparent three dimensional movement of the picture 
as defined by the operator is stored as a sequence 
of address signals defining how each frame of the 
video picture sequence will be manipulated to 
represent the movement. Movements are usually 
required to be smooth but not all addresses, either 
spatially or temporally, need to be stored, because 
Intermediate addresses may be calculated by extra- 



polation. Thus In defining a movement only certain 
addresses for so-called "key" frames, I.e. image 
frames having a position actually defined by the 
operator, need to be stored and intermediate frames 

5 can be calculated by extrapolation between key 
frame addresses. Thus for example, a pendulum 
movement of the picture may be defined by 
movement along the X axis about a higher level 
reference point The positions of maximum displace- 

10 ment in such a move may be defined as key frames 
and intermediate frames may be automatically 
calculated. The pendulum effect is then created by 
running the system and its speed is determined by 
the number of intermediate frames of the input 

15 picture sequence which are* generated between key 
frames. 

For a particular video production, and for future 
general reference, an operator may store a library of 
effects on a disc memory device which provides fast 

SO and substantially random access loading of data to 
the system's transforming circuit 25. Effects are 
difficult to describe and classify In a way which has 
any obvious meaning and therefore a particular 
previously defined effect will usually be arranged to 

25 be located by the user by previewing a number of 
effects from disc. The provision of the graphics store 
27 also allows additional features to be provided 
which are not present in hitherto known systems. In 
accordance with another feature of this embodi- 

30 ment, the system may advantageously be arranged 
to display on the monitor address data In the form of 
a locus which shows the movement of a predeter- 
mined picture element over the duration of the 
effect. The locus, or string, represents the move- 

35 ment over time of the effect and thus may be 
previewed by the user to assist in the identification 
of a previously defined effect 

Figure 3 of the accompanying drawings is an 
exemplary showing of the locus 40 of the centre 

40 point of the picture, i.e. the origin of axes 31, which 
constitutes in this example the abovementioned 
predetermined picture element. Locti be displayed 
representing movement of the origin of each set of 
axes in the corresponding colour associated with 

45 that level. The locus 40 thus may be displayed as 
indicated schematically in Figure 3 as a series of 
segments 41 to 44 etcetera alternating in colour 
between the relevant level colour and another 
predetermined colour, say black. The points 45 to 49 

50 at which segments meet represent the positions of 
frames of the video Image, and the number of 
segments along the locus 40 represents the speed 
of the effect. A relatively targe number of small 
segments will thus represent a slow effect, whereas 

55 a few large segments will represent a fast effect. The 
positions of key frames may be identified by markers 
such as the small lines 50 that are shown in Figure 3 
extending perpendicularly to the locus 40 at the 
relevant positions. 

60 Locil are calculated using the same process that is 
used on each image picture point during the effect. 
As Is known for example from our European Patent 
Application, Publication No. EP-A-02B3159 
aforementioned three dimensional movement of a 

65 picture which gives rise to new co-ordinates (x*,/ jf) 
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data to simulate movement of a video image In 
three dimensional space, and wherein the 
system further comprises means for storing 
data defining the nature of said manipulations, 
means for determining address data represerv s 
tative of the positions of a predetermined 
element of a manipulated video image, and 
means for displaying connections between said 
positions to show the movements of said 
element. 10 
11. A system as claimed in claim 10 wherein 
said user operable means comprises a press- 
ure sensitive stylus and a touch tablet, and the 

15 
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35 



40 



45 



50 



55 



60 



system Is arranged to effect image manipulation 
by display of at least one set of axes on said 
viewing screen and by rotation and/or transla- 
tion of a video image by reference to said at 
least one set of axes under control of said user 
operable means. 

12. A system as claimed in claim 10 or 11 
wherein key positions of the manipulated (mage 
are determined by operation of said user 
operable means and are correspondingly dis- 
played, and positions between succeeding key 
positions are automatically calculated. 
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